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Abstract 

Convex relaxations based on different hierarchies of Unear/semi-definite programs have been 
used recently to devise approximation algorithms for various optimization problems. The ap- 
proximation guarantee of these algorithms improves with the number of rounds r in the hierarchy, 
though the complexity of solving (or even writing down the solution for) the r'th level program 
grows as n^^^^ where n is the input size. 

In this work, we observe that many of these algorithms are based on /oca/ rounding procedures 
that only use a small part of the SDP solution (of size i)2°M instead of n^('')). We give an 
algorithm to find the requisite portion in time polynomial in its size. The challenge in achieving 
this is that the required portion of the solution is not fixed a priori but depends on other parts 
of the solution, sometimes in a complicated iterative manner. 

Our solver leads to n*^'-^^2'^('') time algorithms to obtain the same guarantees in many cases 
as the earlier n^^'^^ time algorithms based on r rounds of the Lasserre hierarchy. In particular, 
guarantees based on O(logn) rounds can be realized in polynomial time. For instance, one can 
(i) get 0{1/Xr) approximations for graph partitioning problems such as minimum bisection and 
small set expansion in n^^^^2'-'^^^ time, where is the r'th smallest eigenvalue of the graph's 
normalized Laplacian; (ii) a similar guarantee in n^^^^ k'^^'^'> for Unique Games where k is the 
number of labels (the polynomial dependence on k is new); and (iii) find an independent set of 
size fl{n) in 3-colorable graphs in (n2'")'^(^^ time provided Xn-r < 17/16. 

We develop and describe our algorithm in a fairly general abstract framework. The main 
technical tool in our work, which might be of independent interest in convex optimization, is 
an efficient ellipsoid algorithm based separation oracle for convex programs that can output a 
certificate of infeasihility with restricted support. This is used in a recursive manner to find a 
sequence of consistent points in nested convex bodies that "fools" local rounding algorithms. 
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1 Introduction 



A rich body of recent research has shown that for many optimization problems, the Unique Games 
conjecture (UGC) serves as a barrier to further improvements to the approximation factor achieved 
by efficient algorithms. In many cases, including all constraint satisfaction problems and various 
graph partitioning problems, the best algorithms are based on fairly simple semi-definite program- 
ming (SDP) relaxations. The UGC foretells that for these problems, no tighter relaxation than 
these simple SDPs will yield a better approximation ratio in the worst-case. 

Hierarchies of convex relaxations. A natural question thus is to understand the power and 
limitations of potentially stronger SDP relaxations, for example those from various hierarchies 
of relaxations. These hierarchies are parameterized by an integer r (called rounds/levels) which 
capture higher order correlations between (roughly r-tuples of) variables (the basic SDP captures 
only pairwise correlations, and certain extensions like triangle inequalities pose constraints on 
triples). Larger the r, tighter the relaxation. The optimum of n'th level of the hierarchy, where n 
is the number of variables in the underlying integer program, usually equals the integral optimum. 

There are several hierarchies of relaxations that have been studied in the literature, such 
as Sherali- Adams hierarchy of linear programs [SA90], the Lovasz-Schrijver hierarchy [LS91], a 
"mixed" hierarchy combining Sherali- Adams linear programs with the base level SDP, and the 
Lasserre hierarchy [Las02] (see [CTll] for a recent survey focusing on their use in approximate 
combinatorial optimization). Of these hierarchies, the most powerful one is the Lasserre hierarchy 
(see [Lau03] for a comparison), and therefore holds the most potential for new breakthroughs in 
approximation algorithms. Arguably, Lasserre SDPs pose the currently strongest known threat to 
the Unique Games conjecture, as even the possibility of the 4'th level of Lasserre SDP relaxation 
improving upon the Goemans- Williamson 0.878 approximation factor for Max Cut has not been 
ruled out. Recently, it has also been shown that 0(1) rounds of the Lasserre hierarchy are able to 
solve all candidate gap instances of Unique Games [BHK'^12]. (On the other hand, for some of the 
weaker hierarchies, integrality gaps for super-constant rounds are known for various Unique-Games 
hard problems [KS09, RS09].) 

In light of the above, the power and limitations of the Lasserre hierarchy merit further inves- 
tigation. There has been a fair bit of recent interest in Lasserre hierarchy based approximation 
algorithms [CS08, KMNIO, GSll, BRSll, RT12, AGll, GS12a]. For instance, our work [GSll] 
shows that various graph partitioning problems (including minimum bisection, sparsest cut, and 
Unique Games) can be well-approximated by ~ r rounds of the Lasserre SDP on graphs whose r'th 
smallest eigenvalue (of the Laplacian) is reasonably large. 

A (near)-optimal solution to the r'th level Lasserre relaxation can be found in n'^^^^ time. So 
understanding the power of these relaxations for small values of r is of particular interest. The main 
contribution of this work is to improve the running time of various Lasserre-based approximation 
algorithms to 2'-^^^^n^^^^ (from the default n^^'^^). In particular, the guarantees achieved by O(logn) 
rounds of Lasserre SDPs can be realized in polynomial time. Plugging our methods into some of 
the algorithms in [GSll] gives us the following application: 

Theorem 1. For the graph partitioning problems such as uniform sparsest cut, small set expansion, 
and minimum bisection, one can compute a cut with cost at most ^.^j'^^^ | times the optimum 

in time, where n is the number of vertices and Xr the r'th smallest eigenvalue of the 

normalized Laplacian of the input graph} 

^For the case of minimum bisection, the cut may have a o(n) imbalance. In fact, an approximation factor of 
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For Unique Games with k labels and n variables, an approximation factor 0/ 3/ min{l, A^} can 
be achieved for the minimization version in time n^^^^k^^'^\ where Xr the r 'th smallest eigenvalue of 
the normalized Laplacian of the constraint graph. (Note the polynomial dependence of the runtime 
on the number of labels, which is new to this work.) 

Our methods also apply to Lasserre hierarchy based approximation algorithms discovered re- 
cently. A table of several algorithms whose algorithms we are able to improve is given in Section 7. 
In particular, we describe how Lasserre hierarchy based algorithms for finding independent sets in 
3-colorable graphs [AGll] and approximating 2-CSPs [BRSll] can be fit into our framework. The 
details of this are described in Appendix B, and along the way, in Appendix A, we collect useful 
geometric bounds on the variance of conditioned solutions in Lasserre SDPs. As a result, we get 
the following speed-ups to the algorithms in [BRSll, AGll]. 

Theorem 2 (Informal). Given a 3-colorable regular graph G, we can find an independent set of 
size at least n/12 in 2^^^''^n^^^^ time when the r'th largest eigenvalue of the normalized Laplacian 
of G is at most 17/16. 

Theorem 3 (Informal). Let V be an arity 2 CSP over a domain of size k, and let e £ (0,1). 

Given an instance of V whose constraint graph G is regular, we can find an assignment satisfying 

at least opt(G') — e fraction of constraints in 2^'P°^y('^/^)nP°^^^^/^-' time assuming that the r 'th smallest 

2 

eigenvalue of the normalized Laplacian of G is at least 1 — (Here opt(G) denotes the fraction 
of constraints of G satisfied by an optimal assignment.) 

Our techniques might also be useful in the context of fixed-parameter tractability, which we 
leave as a potentially interesting avenue for future research. 

Local rounding algorithms. Note that even writing down the full r-round Lasserre solution 
takes n^^''^ time. The hope to speed-up the algorithms to a runtime dependence of 2*^^^^ is based 
on the observation that many of the rounding algorithms have a "local" character that uses only 
a small portion of the SDP solution. In the simplest setting, the rounding algorithm proceeds in 
two steps: (i) find a "seed set" S* oi ^ r nodes based only the solution to the base (1-round) SDP, 
and (ii) use the value of r-round Lasserre solution on the set S* to sample a partial assignment to 
S* and then propagate it to the other nodes. Thus the rounding algorithm only uses the portion 
of the Lasserre SDP corresponding to the subsets S* U {u} for various u. Further, the analysis of 
the rounding algorithm also relies only on Lasserre consistency constraints for subsets of S* . More 
generally, the algorithms might pick a sequence of seed subsets Si, S2, ■ ■ ■ , iteratively and the 
SDP solution restricted to subsets of 5i U 5*2 U • • • U 5^ is used for rounding. 

Note that the needed portion of the solution (corresponding to S* , or more generally Si U 
52 U • • • Si) itself depends on certain other parts of the solution. So one cannot simply project the 
space down to the relevant dimensions to find the required part of the Lasserre solution. Our main 
technical contribution is an ellipsoid algorithm that can find the needed partial solution (which 
satisfies all the local constraints induced on those variables) in time polynomial in the number of 
variables in the partial solution. We stress that the partial solution we find may not extend to a 
full Lasserre solution. This, however, does not matter for the approximation guarantee as it will 
"fool" the rounding algorithm which can't distinguish the solution we find from a global Lasserre 
solution. 

There are two examples of hierarchy based approximation algorithms which have been sped 
up to 2^^^'^ dependence on the number of rounds, both of which rely on weaker hierarchies than 

. can be achieved in n°(^''=''2°(''/'=) time. 
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Lasserre: (i) the algorithm for sparsest cut on bounded treewidth graphs using the SheraU- Adams 
hierarchy [CKRIO] and (ii) the Unique Games algorithm based on the "mixed" hierarchy [BRSll]. 
The faster algorithm is for the former case is immediate as the required portion of the solution only 
depends on the input graph, so one can simply find that part using any LP solver. For the Unique 
Games algorithm, the seed set S* depends on the vector solution to the basic SDP relation. The 
goal is to extend the solution to local distributions of labels on subsets S* U {u} for various nodes 
u, whose 2- way marginals agree with the vector inner products. As briefly sketched in [BRSll], 
these constraints form a linear program, and if infeasible, by Farkas' lemma, one can get a new 
constraint for the vector inner products, which can be fed into an ellipsoid algorithm for solving the 
basic SDP. Our situation is more complicated as we handle several iterations of seed set selection, 
and the "extension" problems we solve are no longer simple linear programs. Also, the runtime of 
the Unique Games algorithm in [BRSll] had an exponential dependence on the number of labels, 
as opposed to our polynomial dependence. 

Main technique: Separation oracle with restricted support. We describe the high level 
ideas behind our method for finding adequate partial solutions to Lasserre SDPs in Section 2. Our 
approach applies in a fairly general set-up, and therefore we describe our methods in an abstract 
framework for clarity, both in Section 2 and later in Section 4 where the formal details appear. 

In addition to the runtime improvements, our results contribute a useful, and to our knowledge 
new, basic tool in convex optimization, which is an efficient ellipsoid algorithm based sep- 
aration oracle that can output a certificate of infeasibility with restricted support (or 

more generally belonging to a restricted subspace). For instance, suppose we are given a convex 
body K C M" via a separation oracle for it. Given a point y S (a potential partial solution) for 
some U C {1, 2, . . . , n}, we give an algorithm to either find x £ K such that proj[;(x) = y (if one 
exists^), or find a separating constraint that is supported on U. 

1.1 Organization 

Our paper is organized in the following way. In Section 2, we formalize the notion of local rounding 
algorithms and briefiy introduce our main technical contribution. We then present some mathe- 
matical preliminaries in Section 3 and our main technical contribution in Section 4. Our final solver 
is given in Section 5. 

In Section 6, we introduce Lasserre Hierarchy in a way suitable for our framework. Finally 
we state the faster running times we obtain for various rounding algorithms based on Lasserre 
Hierarchy in Section 7. 

For the semi-coloring algorithm from [AGll] and the 2-CSP algorithm from [BRSll], we present 
the faster rounding algorithms and prove their correctness in Appendix B. Along the way, in Ap- 
pendix A, we give a geometric interpretation of variance reduction via conditioning as needed in 
[BRSll, AGll], using terminology and methods from [GSll]. We believe this could be useful 
elsewhere. 

^Actually, we need the volume oi K D proj|^^(j/) to be at least some small e 
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2 An abstract framework of local rounding and Overview of our 
techniques 



Consider a rounding algorithm with foUowing property: Given an optimal solution x G M as 
input, it only reads a much smaller part of this solution, say T Q N with |r| = o(|A^|). We call 
these "local" rounding algorithms: Even though this setting might sound too restrictive and/or 
unrealistic, observe that several of the known rounding algorithms which use "hierarchies" fit into 
this framework, [CS08, KMNIO, GSll, AGll, RT12, GS12a]. See Section 7 for details. 

Local Rounding. We first start by outlining a generic iterative rounding algorithm. This frame- 
work depends on two application specific deterministic^ procedures, SEED and FEASIBLE. Without 
going into the formal details, at a high level, SEED5 procedure chooses next "seed set" designating 
which fragment of the solution we will read based on current seeds S and FEASIBLE5'(7/) is a strong 
separation oracle for a convex body Ks representing the induced solutions on seeds S. 

At the end, final seeds and induced solution are fed into another application specific rounding 
procedure. 

Formal Framew^ork. Given two problem specific procedures, SEED and FEASIBLE, we formalize 
the generic algorithm described above as follows. 

1. Let X G be a vector representing an optimal solution for some convex optimization 
problem, x £ Kjy. 

2. Let 5(0) be the initial solution fragment and y(0) ^ xs(o) be the induced solution. 

3. For i ^ to t 

(a) Fail if FEASIBLE5'(j)(y(i)) asserts infeasible (i.e. y{i) ^ ^^(j)). 

(b) U i < £, read next part of solution: S{i + 1) SEED s(^i-^{y(i)) and y{i + 1) xs(i+i)- 

4. Perform rounding using S{£) and y{i). 

Our Goal. Suppose |5(^)| ^ |A^| - the algorithm reads only a negligible portion of the full 
solution. Then can we find an equivalent rounding algorithm which runs in time poly(|S'(^)|) as 
opposed to poly(A^)? Claim 4 shows this can be expected: 

Claim 4. Above rounding algorithm can not distinguish between the following two cases, i.e. any 
properties satisfied by the output assuming 1 still holds under a weaker condition, 2: 

1. There exists a feasible solution x G M^, i.e. FEASIBLE]\f{x) asserts feasible. 

2. For allie {0, ...,^}.- 

• y{i) £ Kg(^^y FEASIBLEg(^^i^{y{i)) asserts feasible, 
. S{i + l) = SEEDs(i){yii)) ifi<l, 

• yii + 1)5(0 = y^^) ifi<l. 

Using this insight, we first consider a simple case and give an algorithm whose running time depends 
on \S{1)\ instead of |A^|. 

^We can allow randomization also, but we stick to the deterministic case for simplicity, since all the seed selection 
procedures used by the known algorithms can be derandomized. 
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2.1 An Algorithm for a Simple Case 

Suppose that SEED procedure does not depend on y. Then the above conditions can easily be 
expressed as a convex problem of size |5'(^)|, which is much smaller than the original problem. 
Then we can solve this convex problem using standard ellipsoid procedure and execute the above 
procedure on this solution instead. 

2.2 Our Algorithm 

Unfortunately for all algorithms we consider in this paper, the procedure SEED heavily depends on 
y. In particular, at the i*'* level, ^ i < £, we are trying to solve the following induced problem on 
S{i + 1). Given y{{) G Kg^i): 

Find y{i + 1) 

St y{i + i)s(i) = y{i), y{i + i) g ^s(i+i); 

3y{i + 2) E Ks^i+i) : y(i + 2)s(^+l) = y{i + 1) 

where S{i + 2) = SEED s^i+i){y{i + 1)); (i) 

G Ksi^,) : y{^)s{t-i) = y{i " 1) 

where = SEED5(,_i)(y(£ - 1)). 

Observe that if we can construct a weak separation oracle for eq. (1) at (i + 1)*'^ level, then we can 
combine it with ellipsoid algorithm to solve the problem at i^^ level also. Thus if we can convert this 
ellipsoid algorithm to a weak separation oracle, then we can call these separation oracles recursively 
starting from O*'* level all the way down to C''^ level: 

Recursive Separation Oracle. (Template for i*^ level) 

1. Given S{i) and y{i), if FEASIBLE5(j)(y(i)) asserts infeasible and returns c, then assert infea- 
sible and return c (to the {i — 1)*'^ level). 

2. If i = i, then return the solution y{i). 

3. Let S(i + 1) ^SEED5(,)(y(i)). 

4. Use ellipsoid method to find y{i + l) such that y{i + l)j = y{i)j for all j G S{i) with separation 
oracle being a recursive call for the {i + I)*'* level (which takes inputs S{i + 1) and y{i + 1)). 

5. If ellipsoid method fails to find such solution y{i + 1), return a separating hyperplane. 

The key question now is how one might implement (the currently vague) step 5. Let us inspect a 
simple option, and see what goes wrong with it. 

Return an arbitrary hyperplane seen so far. Any inequality returned by the recursive sep- 
aration oracle call is a valid separating hyperplane, so consider returning an arbitrary one. What 
goes wrong in this case? The problem is that the running time now might be as large as polynomial 
in |A^|. To see this, suppose that FEASIBLE5(£)(y(£)) returned an inequality on support S{£). Then 
the parent ellipsoid procedure needs to keep track of the additional variables from this particular 
S{1), call it S. At some later stage, the algorithm may backtrack and change an earlier seed set, 
say S{i — 5), which will need to a new S{i). But the algorithm would still need to keep the values 
of variables from the S, the old value of S{£). Continuing in this fashion, the set of variables the 
algorithm has to track might end up being A^, which is equivalent to constructing the whole solution 
on M^! 
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This attempt has not been futile though, as it shows what kind of hyperplanes we need: 



Any hyperplane returned by step 5 at {i + ly^ level should have support S{i). (2) 
We outhne our proposed solution in the next section. 

2.3 Our Contribution: A Separation Oracle with Restricted Support 

Our solution to 2 is based on a new ellipsoid algorithm for finding separating constraints with 
restricted support. Specifically, the main technical contribution of this paper is Algorithm 1 with 
the following guarantee: Given a feasibility problem of the form 

Find y G M" subject to Uy = yo, y £ mt{K), 

where 11 is a projection matrix, yo G span(n) C R"; along with separation oracle for convex body 
K; it either finds feasible y or asserts that the problem is infeasible and outputs a separating 
hyperplane c G span(n). This algorithm coupled with the recursive separation oracle meets both 
our correctness and running time requirements. In particular, the running time instead of being 
the trivial bound of |A^|*^(^) will be roughly \S{£)\^^^\ Assuming the exponential-time hypothesis, 
the exponential dependence on the number of seed selection stages £ cannot be avoided (a sub- 
exponential dependence would lead to a f{k)n''^^^ time algorithm to decide if an n-vertex graph 
has a /c-clique). 

Remark 1. Our algorithm can be thought as a weak separation oracle for eq. (1) at level i given 
a weak separation oracle for level i + 1. When each convex body Kg(2)^ ■ ■ ■ i^sii) is guaranteed 
to be a polytope, such as LPs from the Sherali-Adams Hierarchy, it is known that one can obtain 
a strong separation oracle at level i by using only using a strong separation oracle at level i + 1 
(see Corollary 6.5.13 in [GLS93]). However in the case of semi-definite programming, it is an open 
question [PK97] whether one can obtain a strong separation oracle from another strong separation 
oracle in polynomial time. 

3 Preliminaries 

For any positive integer n, let [n] = {1, 2, . . . , n}. We will use to denote empty set. Given set A, 
let 2^ be its power set, i.e. set of all subsets. For any real k, we will use A^, A^k and A^k to denote 
the set of all subsets of A having size exactly k, at most k and at least k respectively. Observe that 
2^ = A^Q, = Finally note that A^i is the set of non-empty subsets of A. 

Given sets A, B and a field F (M for reals, Q for rationals) , we will use and F^^-^ to 
denote vectors and matrices over F whose rows and columns are identified with elements of A and 
B respectively. For any function / : ^4 — )• F (resp. 51 : j4 x i? — )• F), we will use [f{u)\u^A (resp. 

t')](u^^)g^x_B) to denote the vector (resp. matrix) whose value at row u (resp. row u and 
column v) is equal to f{u) (resp. g{u,v)). Given vector x G F^ and matrix Y G F"^^^, for any 
subset C and D let xc G F*-" and Yc^d G F*^^^ denote the minors of x, Y on rows A n C and 
columns B D D with O's everywhere else. 

Finally we will use S"^ D D S^_,_ to denote the set of symmetric, positive semi-definite and 
positive definite matrices on rows and columns A. 
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3.1 Convex Geometry and Ellipsoid Method 

The main crux of our algorithm rehes on an ehipsoid solver method which can also return a 
certificate of infeasibility. Throughout this section, we assume the underlying space is n-dimensional 
whose coordinates are identified with [n]. So all our vectors and matrices (unless noted otherwise) 
will have [n] as their rows. 

Notation 5 (Projection). We will use H £ s{j!^ to denote a projection matrix representing some 
linear subspace span(n) C and H'^ to denote the projection matrix onto null space ofH, i.e. 
= identity - 11. 

Given vector yo G M'"', we will use if yo is in the span ofH, i.e. Hyo = yo and we will 

use n~^(yo) to denote the following set of vectors: 



n-i(yo) = {ye 



n 



n^(y-yo) = 



Notation 6 (Balls). Given a set K C rN and non-negative real e ^ 0, we define M{K, ±e) in the 
following way. 

M{K,e) = jx e R["l|3y G K s.t. ||y-2;||2 ^ e} . 

M{K,-e) ^ K\]B(r["] \i^,e). 

Observe that for y G R["1, M(y,e) is the n-dimensional sphere with origin y, with M{K,e) being 
Minkowski addition of sphere of radius e to K and M{K, — e) being Minkowski subtraction of sphere 
of radius e from K . 

Observation 7 (See [GLS93]). For any convex body K C Rt"! and non-negative reals £,ei,e2, the 
following hold: 

1. M{M{K, e), -e) = K, M{M{K, -e),e) C K. 

3. M{M{K, -ei), -ea) = IB(i^, -ei - ea). 

Notation 8 (Volumes). Given K C rM, we will use \o\d{K) to denote d-dimensional volume of 
K, provided it exists. Furthermore for any non-negative real e ^ 0, let volrf(e) be the volume of 
d-dimensional ball of radius e. We will use yoY^'^{K) to denote the radius of a d-dimensional sphere 
whose volume is equal to vold(i^) so that 

yoUK)=yoUyo[2\K)). 

Notation 9 (Polytope). Given matrix A e R™^" and vector b G R™, let 

poly(yl,6) = |x G R["1|^j; ^ 6} . 

Lemma 10 (See Lemma 3.2.35 in [GLS93]). Given a polytope P = poly(74, b), for any positive real 
e > 0, 



M{P, -e) = poly(A, b - s^diagiA^A)). 

Here \J diag(yl'^A) denotes the vector whose i*^ coordinate is equal to Euclidean norm of i^^ row of 
A. 



8 



Definition 11 (Separation Oracle). Given a convex body K C rW, SEP^iy) is a separation oracle 
for K if the following holds. On inputs a rational vector y £ Q^"] and rational number 5 > 0, 
SEPs{y) asserts feasible if y £ K . Otherwise, if y ^ K, it returns c such that \\c\\oo = 1 and 

\fx e K : {c,x) ^ {c,y) + S. 

We will use T{SEPs) to denote the worst case running time of SEP 

Theorem 12 (Central-Cut Ellipsoid Method [GLS93]). There exists an algorithm, called the 
central-cut ellipsoid method, 

CCUT-E{SEPs,U,yo,eo) 

that solves the following problem. Given a projection matrix H G s[j^' of rank m, vector G IT, a 
convex body K C [—A, A]["] for some positive A with SEPs (see Definition 11) and rational number 
eo > 0, it runs in time 

I log A|A^ [poly(n) + T{SEP2-n)] where N ^ 6(n — m)(| logeol + {n — m)); 

after which it outputs: 

1. Either a vector a G Q^"] such that a£ K nU^^{y'^); 

2. Or a polytope of the form P = poly(C,(i), where C G Q[^]^W,d G Q'^I with K C P and 
voln-m (Pnn-i(yO)) <eo. 

Proof. Such algorithm can be obtained by trivial modifications to the central-cut ellipsoid algo- 
rithm [GLS93], which we outline here. Handling the constraint Ha = y^ can be done by projecting 
the covariance matrix of ellipsoid onto 11^^ (y^). At /c*'* iteration, for all k, we add hyperplanes 
returned by SEP^ to P. 

Algorithm terminates with a feasible a G Qt"^ with Ha = y^ only if SEP5(a) asserts feasible for 
some 6 ^ Eq, in which case a € K. Otherwise, when the maximum number of iterations is reached 
we simply return. □ 

4 Finding Separating Hyperplanes on a Subspace 

We now describe our main technical contribution: An ellipsoid algorithm which can output a 
certificate of infeasibility on a restricted subspace using only the separation oracle SEP^ as in 
Definition 11. The procedure uses the central-cut ellipsoid method [GLS93] as a sub-routine. The 
main technical ingredient of our algorithm is Theorem 17, which is stated and proven in Section 4.1: 
It allows us to express this as another convex programming problem in terms of the "history" of 
constraints returned by separation oracle. Finally in Section 4.2 we present our ellipsoid algorithm, 
bound its running time and prove its correctness. 

4.1 An Equivalent Convex Problem 

We first state some useful propositions. Recall our goal: Given convex body K, a subspace 11 
and yo G H, we have a polytope P separating various points {y} C 11^^ (yo) from K. We want 
to compute a separating hyperplane on 11. Our approach is formulated in Lemma 16, see also 
Figure 1. We first show that points interior in K have far off projections from Hyo- 
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Lemma 13. Given convex body K C rN^ a projection matrix 11 G s[[^ with rank(n) = m, vector 
yo G RI"] and positive real 6 > vol~l^ (ll~^{yo) H K) , 

for aUyeM{K,-26), ||n(y - yo)|| ^ 

We can restate our goal as the following: Given convex body K, a subspace 11 and yo G 11, we 
have a polytope P separating various points {y} C n~^(yo) from K. We want to show that points 
interior in K have far off projections from Ilyo- First, we show that points in the interior cannot 
project exactly onto Hyo- 

Proposition 14. Given K C rN^ a projection matrix 11 G s[j^' with rank(n) = m, and y G R^"' 
the following holds: For any 6 > vol^^^(n~-'^(y) n K), 

n-^{y)nM{K,-6) = 0. 

Proof. If 3x G I[~^{y) n M^K, —S), then Observation 7 implies B(x, 6) C K. In particular, 

]B(x,5)nn-i(y) cn-i(y)nK =^ volm{U-\y)nK)^yol^{U-\y)nMix,5)). 

Finally since x G Il~^{y), M{x,6) H n~^(y) is an m-dimensional ball of radius 6, whose volume is 
volm((5) > volm(n"^(y) Pi K). Hence 

voim(n-Hy) n K) ^ volm {M{x,6)nu-\y)) >voim{'n-\y)nK), 

which is a contradiction. □ 

Lemma 13 is simply a quantitative version of the above, showing that points further interior in 
K have far off projections from Hyo- 

Lemma 15 (Lemma 13 restated). Given convex body K C r["1^ q projection matrix 11 G s[[*^ with 
rank(n) = m, vector yo G R'"^ and positive real 5 > vol^l^ (n^-'^(yo) H , 

for aUy€M{K,-26), ||n(y - yo)|| ^ 

Proof. For the sake of contradiction, assume there exists y G M{K, —2S) such that ||n(y — yo) \\ < 5. 
Consequently liM^y, 6), which is a sphere of radius 5 on span of 11, contains Ilyo- In other words, 

0/n-i(yo)nB(y,5). (3) 

Since y G M{K, —25), by convexity of K, we can repeatedly apply Observation 7 to show that 

y eM{K, -26) = M(M(K, -6), -6), 
B(y, 6) CB(B(B(K, -<5), -6), 6) C n{K, -6). 

Substituting this into eq. (3), we have / Il^^{yo)r]M{y, 5) C n^^(yo)nB(i^, —5) which contradicts 
Proposition 14 for our choice of 5. □ 

Having shown that there is a 5- neighborhood of Hyo disjoint from interior of K whenever their 
intersection has small volume, we can immediately use Minkowski's Separating Hyperplane The- 
orem to infer the existence of such hyperplane. In fact, any hyperplane perpendicular to the line 
from yo to the closest point in K has this property. We formalize this below. 
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Figure 1: We want to find a hyperplane parallel to 11 separating n~^(yo) and K, using only the 
inequalities returned by separation oracle, polytope P. The optimal solution of Lemma 16 is given 
by y* with corresponding hyperplane Il~^{y*). 



Lemma 16. Given convex body K C rN^ a projection matrix 11 G s[|^' with rank(n) = m, vector 
yo G M'"] and positive real 6 > vol^i^ (n~^(yo) H K) , the hyperplane perpendicular to the projection 
of direction from y to closest point in the interior of UK separates yo and interior ofllK: 

Formally any optimal solution y* to the following eq. (4); 



satisfies eq. (5); 



Minimize ||n(y — yo)|P 
subject to y£M{K,-25), 



min (Il{y* - yo),x- yo) ^ ||n(y* - yo)f. 

xeM(K,-2S) 



(4) 
(5) 



Proof. By contradiction. Assume for optimal solution y* , there exists x £ M{K, —26) such that 

(n(y* -yo),x- yo) < My* - yo) f = {U{y* - yo), n(y* - yo)) = (n(y* - yo), y* - yo) . 
Therefore 

(n(y*-yo),a;-y*) <0 (6) 

For some 9 G (0, 1] to be chosen later, consider y{6) (1 — ^) • y* + • x, which is always feasible 
for eq. (4) as x G M{K, -26) and M{K, -26) is convex. Then: 



1 d\\U{y{9)-yo)r 

2 de 



9^0+ 



n(y(^)-yo),n- 



de 



n(yW-yo). 



d{y{0) - yo) 

89 



9^0+ 



e-5>o+ 



{Uiy{9) - yo), X - y*)|,^o+ = (n(y(0) - yo),^ - y*) 
{U{y* -yo),x-y*) <0 
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where we used eq. (6) at the last step. We arrive at a contradiction by noting that above inequaUty 
imphes existence of 9* £ (0, 1] such that 



\n{y{9*)-yo)f <my* -yo)f, yie*) gM{k,-26). □ 

n(y*-yo) 

my*-yo)\\c 



Given Lemma 16, we can choose our separating hyperplane c as c = —Tmj^rzr^li — • But this 



does not quite work for two reasons: 

1. Hyperplane c only separates "strict interior" of K as it is, whereas we need to separate K 
itself. 

2. Depending on K, it might not be possible to represent optimal c using polynomially many 
bits, thus we need to account for near optimal solutions. 

We now show how to overcome these problems. 

Theorem 17. Given convex body K C [—A, A]" for some A > 0, a projection matrix H G s[[^' with 
rank(n) = m, a vector yo G [~^-, ^]"; for any positive real (5 > with 5 > vol~l^ (n~^(yo) H if), 
the following holds: If y' is an jK^^- approximate solution to eq. (7) 

Minimize ||n(y — yo)|P 

(7) 



subject to yeM(^K,-(^2 + ^J^^ ■ 6^ 



then n(y' — yo) 7^ and for c being 

n(y' - yo) 



C : 



|n(y'-yo)||c 



yx£ K : (c, x) ^ (c, yo) + 25y/m- (8) 



Proof. Before we begin, we set e = • ^* optimal solution of eq. (7) with \\y* —y'\\ ^ 

£5. Since y* G B{K, -(2 + e)6), we have B(K, -26) ^ B{y*,e5) 3 y' . By Lemma 13, this implies 
l|n(y' — yo)|| ^ ^1 proving Jl{y' — yo) 7^ 0. For any x £ K, we can decompose x as a; = x' + 2; for 
some x' G B(K, -26) and ||z||2 ^ 26. Then: 

{U{y' -yo),x- yo) ={n{y' - yo),x' - yo) + {U{y' - yo), z) 

^{U{y' - y*),x' - yo) + {U{y* - yo), x' - yo) - \\z\\ • ||n(y' - yo)|| 
^ - ||n(y' - y*)|| ||n(x' - yo)\\ + 5^ -25||n(y' - yo)\\ 
^2Av^ Lemma 16 

^ - 26\\U{y' - yo)\\ (by the choice of e = ^^^) 
^-26V^\\U{y' -yo)\\oo- 
Since c = — \\u(ySy^°)\\ — > 'we have (c, x) ^ (c, yo) + 26^Jrn for any x £ K. □ 

4.2 Ellipsoid Algorithm with Certificate of Infeasibility 

Our solver is given in Algorithm 1. 
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Algorithm 1 CERTIFY-E(SEP5, 11, yoi ^o): Ellipsoid method with certificate of infeasibility. 

Input. • Convex body K C [0, 1]" and separation oracle SEP^ as in Definition 11, 

• Projection matrix 11 G with rank(n) = m, yo ^ RI"! and positive real < eq < 1. 

Output. • Either a vector y G Q"- st y G i^n ^"^(yo)• 
• Or c G n with ||c||oo = 1 and \/x £ K : {c,x) ^ Eq + {c,yo). 

Procedure. 1. Run CCUT-E(SEP5, 11, yoi where e voln-m (2^™ 

2. If it returns y £ K Ci n~^(yo), then return y. 

3. Else let P = poly(C, d) be the polytope it returns. Set 6 ^ ^' ^ 2a'^ ™^ 

4. Solve eq. (9) using regular ellipsoid method to find an e'5-approximate solution, y* G 



Minimize ||n(y - yo)|| subject to Cy ^ d - (2 + e')SJdiag{C^C). (9) 



5. Return - 

miy -yo)\\c 



The proof of the following theorem follows by combining various ingredients we have so far, 
especially Theorems 12 and 17. 

Theorem 18 (Main technical tool). Algorithm 1 runs in time N ■ T{SEP2-n) + poly(n) log^ ^ 
where N = O (^{# of free variables)^ log # °/ fixed varzabies \^ ^ q (^^^ _ ^ provides the 

following guarantee: If vo\^^,^{K D Il~^(yo)) > then it outputs y £ K D n^"'^(yo)- 

Proof of Running Time. By Theorem 12, step 1 finishes in time A^(poly(n) + T(SEP2-iv)) with 
N = 0{{n — nif + {n — m) log 1/e), where log 1/e = log 1/ voln_m(eo/2-y/m) ^ O {{n — m) log 

so A^ = O [{n-mf\ogf^. 

For step 4, we can implement a simple separation oracle which runs in time 0{Nn). The regular 
ellipsoid method requires 0(n^ + nlog(l/e'5)) iterations to reach an accuracy of e'6. Each iteration 
takes time poly(n) in addition to separation oracle, therefore the total running time of fourth step 
is bounded by A^poly(n) log(l/eo) = poly(n) log^ Hence the claim follows. □ 

Proof of Correctness. There are two cases. If algorithm outputs y at step 2, by Theorem 12 y £ 

Q"nn-i(yo)nif. 

Now consider the other case. Then step 1 will output a polytope P = poly(C, d) such that 
K P, whose volume is bounded by 

vol„_m(Pnn-^(yo)) < vol 

n—m 

The set of feasible solutions for eq. (9) is B(P, -{2+e')6) by Lemma 10. Since vol;;l„(Pnn"^(yo)) < 
6 := we can apply Theorem 17 and conclude that c as constructed in step 5 will have the 

following properties: 

• c G n. 



(^) 
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• For all X G P, (c, x) ^ (c, yo) + '^\/rn5. To see this, note K P, for all x £ K means: 

(c, x) ^ max(c, x) ^ max(c, x') ^ (c, yo) + l^pmb = (c, yo) + ^o- 

This finishes the proof of correctness. □ 

5 Faster Solver for Local Rounding Algorithms 

We return back to our motivating example. Assume we have n variables, and we want to find a 
discrete labeling x G LI"! of those from a set of labels L, under various constraints and objective. 
Suppose we "lifted" this problem into a higher dimension where | n, and obtained a family 
of increasingly tight convex relaxations defined over various subspaces of . 

Formally, we have a set of subspaces {n5}5c[Af]) represented by their projection matrices and 
associated with subsets of [A^], and with each subspace Us, we have an associated convex body, 
Ks CUs[0,l]^ with such that 

n5 C Hr =^ UsKt c Ks- 

We are given functions FEASIBLE, ROUND and SEED, along with positive integers n,s such 
that: 

• FEASIBLE^ : n^Q^ ^ {feasible, HsQ^}. On input S O N,y £ UsQ^ , it asserts feasible 
if y e Ks or returns c G n5Q^ : ||c||oo = 1 such that"^ Vx G Ks ■ (c, x) < (c, y) in time 
poly(rank(ns)). 

• SEED5 : Ks 2^. Given S C [A^] and y E H^Q^, it returns subset S' ^ S such that 
'rank^(ng')^ ^ ^ when S ^ and rank(n5') ^ n when S = $. Its worst case running time is 
bounded by poly(rank(n5)) (or poly(n) in the case of S" = 0). 

• ROUND^ : Ks — >■ L^"!. On inputs S N and y G Ks, returns an approximation to the 
original problem in time poly(rank(n5)). 

We now describe our main solver. Note that once the algorithm outputs y* G Ks(e), the final 
output labeling will simply be R0UND5(£)(y*). The proof of the following theorem is easy given 
the ingredients so far. 

Theorem 19. Algorithm 2 runs in time [s^nlog(l/eo)]'^^^^ (compare this with the straightforward 
algorithm which runs in time A^*^^^-* log(l/eo)J cLnd achieves the following guarantee: Provided that 
vol -fC > So, it outputs y* G AT^^^-j and S{0), . . . , S{i) st for all i: 

n5(,)y* Gi^5(,), (10) 
S{z + l)=SEEDs^,){y*). (11) 

Otherwise it asserts vol A' ^ eq. 

Furthermore there is no algorithm which runs in time n"^^^ assuming Exponential Time Hypoth- 
esis. 



''We can handle FEASIBLEs that only returns a weak separation oracle, but since in our application to SDPs we 
have access to a strong separation oracle, we assume this for simplicity. 
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Algorithm 2 Fast Solver (to fool local rounding algorithm) 
Input. • Maximum number of iterations i and positive real eq > 0, 

• n, r, {Ks)sc[N] with separation oracle FEASIBLE, SEED, 110 and y(0) G KqQ^ all as described 
in Section 5. 

Output. • Either asserts vol-ftT ^ 

• Or outputs y* G and S{0), . . . , S{£) st for all i: 1. Ils{i)y* G Ks^i)'-, 2. S{i + 1) = 
SEEDs(,)(y*). 

Procedure. 1. Initialize global variables 5(1), . . . , S'(£) representing seed sets and global sparse 
vector y* E Qt'^l representing the final solution (it will be in span oiIlg(^(^). 

2. Set S(0) ^ {0}. 

3. Run CCUT-E(SEP5(o),(5i Oi 0) £o) (see Theorem 12) where SEP is given in Algorithm 3. 

4. If it asserts feasible, output 5(0), ... , S{t} and y* . 

5. Else assert vol-fT ^ Eq. 



We prove correctness, running time claim, and ETH hardness in turn. 

Proof of Correctness. First we assume correctness of Algorithm 3 and prove correctness of Algo- 
rithm 2. There are only two cases: 

1. Algorithm 2 returns y* G only if CCUT-Egj-^) returns a feasible solution. For such y* , 
by Theorem 12, SEP5(i) ^^(y*) asserts feasible. By correctness of SEP^^i) ^^jd/*), y* satisfies 
all claims. 

2. Else algorithm asserts vol^jv] K ^ eq which means CCUT-E asserted Eq ^ vol5(o) ^i^s{i)i ^o) ^ 
vol^i^o) -^S(o) • We know that 115(0)-^ ^ -^^5(0) and K C [0,1]'^] therefore vol[^](i^) ^ 
^olUs(o)K ^ vols(o) Ks{o) ^ eo- 

Now we will prove correctness of Algorithm 3 inductively starting from i = 1. For each i, in 
order to prove inductive step, we need to consider in which one of the following steps SEP5(j) 
returned: 

1. Step 1: Follows from definition of FEASIBLE. 

2. Steps 2 and 6: By construction of S{j)'s and inductive hypothesis, S{j + 1) = SEED5(j-)(y*) 
holds for all j ^ i. 

We will prove that FEASIBLE5'(j)(y*) asserts feasible for all j ^ i. For j > i, this immediately 
follows from inductive hypothesis. For j = i, at Step 1 FEASIBLE5(j)(?/*) asserted feasible. 
Thus Ils{j)y* e Ks{j) for ah j. 

3. Step 5: It returns c, only if CERTIFY-E at step 4 outputs c, correctness of which follows from 
Theorem 18. □ 

Proof of Running Time. If we let Ui <— ns^~^ for i ^ 1, we can see that rank(n5-) ^ ni at i*^ 
iteration. Hence 

r(FEASIBLE5(,)) + T(SEED5(,)) = {smf^'^ = nf+«. 
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Algorithm 3 SEP5(j) £p(y):Separation Oracle. 

Input. • Positive real Eq > 0, current iteration i, current seeds S{i), vector y G Q'^^*). 

Output. • Either asserts feasible, and sets values of global variables S{i + 1), . . . , S{£) along with 
y* so that: 

1- ^S(j)y* G for all j : i ^ i ^ £, 

2. S{j + 1) = SEED5(,)(y*) for ah j : f ^ j ^ ^ - 1. 

• Or returns c G ^s{i) with ||c||oo = 1 such that Vx G -^5(i) ■ {c, x — y) < Eq- 

Procedure. 1. If FEASIBLE5'(-j)(y) returns c G H^^j), return c. 

2. Else if i ^ £, set y* ^ y. Assert feasible and return. 

3. S{i + 1)^ SEED si,){y). 

4. Run CERTIFY-E(SEPns(,+,),5,ns(i),y,eo) (see Algorithm 1). 

5. If it returns c, return c. 

6. Else assert feasible. 



If we use Ti to denote the maximum of T{SEP s(i)^£g{y)) over all possible S{i) and y's, with Tq = 
r(main); then Ti = n^^^^ = (r^n)*^*^^) and for any i < £: 

Ti ^0{n^^, logn,/eo)T,+i + ngj^ = {m+ifW log{l / eo)Ti+i = s^Wn^^i) log 1/eo ■ Ti+i 
To=sO(^V('hog\l/Eo). □ 

Proof of ETH Hardness. Consider the A;-clique problem, which cannot be solved in time f{k)n°^^^ 
under the exponential-time hypothesis [LMSll]. Moreover it is easy to see that k rounds of Lasserre 
Hierarchy relaxation is integral for this problem [Lau03], and we can find such a /c-clique using a 
seed selection procedure with £ = k levels. □ 

Next, we will review semidefinite programs from the Lasserre Hierarchy in Section 6 and finally 
in Section 7, we will show a sample of how some approximation algorithms using Lasserre Hierarchy 
relaxation fit into our framework. 



6 Lasserre Hierarchy 

In this section, we present a general derivation of Lasserre Hierarchy relaxation. Although our 
relaxation is only given for 0/1 programming problems, it can easily be adapted to work on any 
set of finite labels. 



6.1 Preliminaries 

For some positive integer d, let Q G Rt"!^'' (resp. V = {Pi, . . . , Pm} C rI'^I*;'') be a degree 
^ d multilinear polynomial (resp. subset of degree ^ d multilinear polynomials) over n variables 
representing objective function (resp. constraints). We want to find a binary labeling of n variables. 
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X G {0, 1}", which satisfies eq. (12): 



Minimize J2se [n] Hues ^« 
subject to EseN^d HneS ^u>^ for all P eV, 
X G {0,1}". 



(12) 



Observe that we can convert any constraint satisfaction problem on {0,1} to this form easily. 
In this section, we will first express eq. (12) as an equivalent SDP problem, from which Lasserre 
relaxation can be obtained by enforcing positivity only on certain principal minors of this matrix. 
This exposition of Lasserre relaxation through constraints on certain minors will be convenient 
when we are presenting our partial SDP solver. 

Notation 20. Given positive integers n,d, for any vector P G RI"!^'' and y G M^'"', we define 
P * ?/ G M^'"' as the following vector: 

{P*y)s = ^ Ptvtus- 

Definition 21 (Multivariate Moment Matrix on V and Q). Given positive integer n let M" : 
" — )• " " be the following linear matrix function: 



'{y) = [yAuB]A, 



BC[n] 



For any V = {Pi, ■ ■ ■ ,Pm} ^ M'"^^'* representing constraints, Q G R^"]^'* and g G M representing 



objective polynomial and a guess for its value, let M"'^ 
following linear function on block diagonal matrices: 



^ ]^([m+2]x2W)x([m+2]x2["l) ^/^^ 



(y) 



r{y) 

qy9-{Q,y) 

M"(Pi*y) 










\P2*y) 








r(PA/*y). 



Having defined the moment matrix, we can state the following: 

Theorem 22 (See [Las02]). eq. (12) has optimal value ^ q if and only if there exists y with y ^ 
such that M"'^'0'«(y) ^ 0. 

Definition 23 (Principal Minors of Multivariate Moment Matrices). Given V C M["kd^ Q g Rb^kd 

and q £ M. as described, for any T : T C 2t"],T D support((5) being a family of sets over [n] 

(y) as the following principal minor of 



containing the support of Q, we define M (y) = M 
M"'^'^''?(y).- 



n,V,Q,q 



T 



T 



(y) 



■(M"(y))y_^ 

92/0 - (Q, y) 

(M"(Pi*y))^y 







(M"(PM*y)) 
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Observation 24. Given a family of sets over [n], T C 2N, for any P € ]R["1=S'^, the minor 
(M"(P * is only a function o/yex(T,d) where 

ex(r, d) = {AU B U C : A £ T, B £ T,C £ [n]^d} ■ 
Proof Consider (M"'(P * y))^ ^ with A,B eU. By Definition 21, this is equal to 

(M"(P * y))^^B = {P * y)AuB = Yl PCVAUBUC, 
where AL) BUC G ex{U, d) by definition. The second part follows immediately from the definition 

n 

ofM ^(y). □ 

Using Theorem 22 and observation 24, we can easily state and prove the following: 

Theorem 25 (Lasserre Relaxation [Las02]). Given positive integers n,r and d, polynomials V C 
I^N^d Qfid Q g M["-kd with q G M, the following is r^^ round Lasserre Hierarchy relaxation of 
eq. (12); 



Find y £ MM^2r+d subject to 



^ ^ {y)hO and 2/0 = 1. (13) 



Note that the straightforward SDP relaxation of eq. (12) corresponds to d rounds of Lasserre 
hierarchy relaxation (and the "basic" SDP relaxation for the case of quadratic polynomials). 

6.2 Separation Oracle for Lasserre Hierarchy 

Given a binary labeling problem, we first cast f rounds of Lasserre Hierarchy relaxation in our 
framework: 

• The set of labels is L = {0, 1}. 

• Lifted space is {^},), the subsets of [n] of size at most r', 

• For any subset S C [n], we define 115 as the projection matrix onto R'^^^'^'^) so that 



[Usx] 



T 



1 ifTGex(5,2), 
else. 



The associated convex body, Ks, is defined as 

Ks = {ye M''^^^'^) : y0 = 1, M |",(5^2)(y) h o} . 

Before stating the FEASIBLE procedure, we need the following well known result: 

Proposition 26. Given a symmetric matrix A G S'^, there exists an algorithm which asserts if 
A'^0 or returns x G such that x'^Ax < in time at most polynomial in size of A. 

Then our FEASIBLE5(y) procedure is trivial: It asserts feasible if M|"^^^2)(y) — 0- Else it 
returns x G Q'^^('^'^) for which x^M |gx(5 2)(^)^ ^ ^' 
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7 Applications of faster solver to Lasserre hierarchy rounding al- 
gorithms 

In this section, we finally apply our main algorithm as given in Algorithm 2 to various rounding 
algorithms for Lasserre Hierarchy relaxations as stated in the introduction. For all these problems, 
our separation oracle is the same procedure as described in Section 6.2. The running times we 
obtained as well as approximation factors and other guarantees are summarized in Figure 2. The 
last two columns list the value of s (the factor by which rank(n5) increases in each step of seed 
selection) and i (the number of iterations of seed selection) used by the rounding algorithm in each 
case. The parameter r refers to the index of the eigenvalue governing the approximation guarantee, 
and e is a positive parameter. 

Algorithm 4 SEED-QIP5(y): Seed selection procedure for approximation algorithms given 
in [GSll] for two-way partitioning problems. 

Input. Subset S CI [n], and y G Q'^^('^'2) provided M ||,x(5 2)(^) — ^ ^^'^ ~ positive integer r' . 
Output. T with |r| ^ r' • IS"]. 

Procedure. 1. Let {xt)t be vectors corresponding to Cholesky factorization of M |^"'^^2)(y)- Foi' 
each 5 G ex{S, 2) and / G {0, l}'^, set 

xsif)^ E (-l)'^'a^/-Hi)uT. (14) 
rc/-i{0) 

2. Let Us ^ T.f e{o,i}s j^;^^jj^xs{f)xs{ff and U^ ^ I - Us. 

3. Use volume sampling [DRIO, GS12b] to choose S': an r'-subset of vectors from (U-gXu{l))^^^^y 

4. Return 5 uy. 



Algorithm 5 SEED-COLORs(y): Seed selection procedure for semi-coloring algorithm as given 
in [AGll] on graph G. 

Input. Graph G on nodes [n], positive integer r' . 
Output. S G [n]<^r'- 

Procedure. 1. Let ^ Y2i=i x^Xu{i) (same embedding as [AGll]). 

2. Use volume sampling [DRIO, GS12b] to choose S, an r'-subset of vectors from {Xu)^,^^,^]- 

3. Return S. 



The claimed running times follow from the ~ g'^^^'^^rfi^^'^ runtime guaranteed by Theorem 19 
for our solver (Algorithm 2). The rounding algorithm in each case runs within the same time. For 
problems marked with *, check the caption for required conditions. 

The works [BRSll] and [AGll] use greedy seed selection, but these can be replaced by the 
above column selection procedure as well. Below, in Algorithms 4 to 6, we present seed selec- 
tion procedures for binary graph partitioning algorithms from [GSll], sparsest cut algorithm from 
[GS12a], and semi-coloring algorithm from [AGll], respectively. 
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Algorithm 6 SEED-SPARSEST-CUT5(?/): Seed selection procedure for sparsest cut algorithm 
as given in [GS12a] on graphs G and H. 

Input. Graphs G,H on nodes [n] with edge weights given by Wu,v respectively, positive integer 
r'. 

Output. S € [?^]^r'- 

Procedure. 1. Let Xu,v ^ \j^^^^v{^u — Xy). 

2. Use volume sampling [DRIO, GS12b] to choose S, an r'-subset of vectors from ^gj^j^- 

3. Return S. 



Problem Name 


Running Time 


OPT 


Rounding 


s 


e 


Maximum Cut 


[GSll] 


20(r/£^)^0{l/£) 


1 — r] 


1-^r? 


20{r/e) 


0{l/e) 


A;-Unique Games 


[GSll] 




1 — r] 






1 


Minimum Bisection 


[GSll] 


20{r/e^)^0(l/e) 


V 


'-^V-o{l) 


20{r/e) 


0(l/e) 


Maximum Bisection 


[GSll] 


20(r/£^)^0{l/£) 


1 — r] 




20ir/e) 


0(l/e) 


Sparsest Cut* 


[GS12a] 




V 




20(r/e) 


1 


Independent Set* 


[GSll] 
[AGll] 


20(r)^0(l) 
20(r)^0(l) 


V 
V 


V 
12 


2U(r) 
20{r) 


0(1) 
1 


Maximum 2-CSPs* 


[BRSll] 


2Poly(fc/e)-r^poly(fc/e) 


V 


r] — e 




{kye^) 



Figure 2: Running times and approximation guarantees for various Lasserre Hierarchy relaxation 
rounding algorithms using our faster solver. For sparsest cut, the spectral assumption is Xr ^ 
r//(l — e). For independent set [GSll], the spectral assumption is Xn-r ^ 1 + 0(1/A) where A is 
the maximum degree. For independent set [AGll], the assumptions are that G is 3-colorable and 
Xn-r ^ 17/16. Finally for maximum 2-CSPs [BRSll], the assumption is that Xr ^ I — (^) . 

We conclude the paper by mentioning some notable Lasserre based approximation algorithms 
for which we are not able to get a runtime improvement: 

• The algorithm for independent sets in 3-uniform hypergraphs [CS08] and the algorithm for 
directed Steiner tree [Rotll], which are adaptive with a large number of stages £ in the 
rounding procedure. 

• The algorithm for minimum/maximum bisection in [RT12], which requires choosing the seed 
set at random independently from the final solution; whereas our solver, by nature, outputs 
a solution which depends on the seed set. 
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A Conditioning and Variance Reduction of Lasserre solutions 

For any 5 C [n], let [k]^ be the set of all possible labelings of S. Recall the vectors xs{f) defined 
in (14). Let [k]'^ = {T} where T denotes the (only) labeling of empty set with 3;0(T) = being 
some constant unit vector. 

Definition 27. Given S C [n] and f G [k]^ with xs{f) 7^ 0, we define the vectors conditioned on 
f as the following. For any A C [n] and g G [k]^, the vector Xji^\j{g) is given by: 

, ^ A XSUAifog) 

Formally the conditional vectors Xji^f{g) correspond to relaxations of respective indicator vari- 
ables. Thus such vectors behave exactly in the same way with non-conditional vectors. Some of 
these properties are given in the following easy claim, whose proof we skip. For g G [k]^ and 
h£[k]^ that are consistent on ^ n we denote by g o h £ [k]"^^^ the labeling that restricts to g 
(resp. h) on A (resp. B). 

Claim 28. For any f G [k]^ with xs{f) 7^ 0, the following are true: 

(a) X0|/(T) = xsifif) and \\xs\fif)\\'^ = 1. 

(b) For any g G [k]^ and h G [k]^ , we have {xA\f{g),XB\f{h.)) = \\XA{jB\f{g°^)\^ if 9,^ are 
consistent on ACi B and otherwise. 
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(c) For any g G [k]^, we have XA{g) = Y.f \\xs{f)\\xA\f{g) so that 

\M9)\? = Y.f\\xs{f)nxA\f{g)\? . 

(d) For any g G [k]^ , WxAyigW = ^'fPsUp^ - 

(e) For any g G [k]-^ and h G [k\^ , 

XAuB\f{g°h) 

Proof. Items a to d are easy. For item e, by definition: 

X ^ xsvjAvjbU ° 9°h) ^ XAuB\f{g°h) ■ \\xs{f)\\ 
''''^^''^ ^ WxsuaU o g)\\ \\xAif{g)\\-\\xs{m ■ 

Assume that some labeling /o G [k]^° to Sq has been fixed, and we further sample a labeling 
f to S with probability (i.e., from the conditional probability distribution of labelings 

to S given labeling /o to ^o). The following defines a projection matrix which captures the effect 
of further conditioning according to the labeling to S. For a nonzero vector v, we denote by v the 
unit vector in the direction of v. 

Notation 29. Given fo G [kf'' and S Q [n], let 

Similarly let Hg = I — Us where I is the identity matrix of the appropriate dimension. 

We will now relate properties of the conditional probability distribution arising from partial 
labelings to the above projection matrix. First we will define the random variables corresponding 
to each indicator function with matching moments: 

Definition 30. Given f G [k]^ , for all g G [A:]"^,/! G [k]^ , let XA\f{g) and X^\j{h) be two random 
variables over {0, 1} such that: 

Proh[XAifig) = lA^Bifih)] = {xA\f{g),XB\f{h)). 

The above definition suggests a very simple rounding scheme: Choose a label for each variable 
based on this probability. In fact, all rounding algorithms we can handle in our framework carry 
this trait. One way to measure how far we can go with only these probabilities is to look at their 
variance: 



Claim 31. \aT{XA\f{g)) = \\xA\f{g)\\ - \\xA\f{g)\\ = \\x^fXA\f{ 
Proof. Since XA\f{g) is a random variable over {0, 1}, 

\^T{XA\f{g)) = IE[^A|/(ff)](l - nXA\f{9)\) = \\xA\f{g)\? - {x^f,XA\f{g)? = ||x^^x^|^(g)f . □ 
Claim 32. Cow {X a\ j {g), XB\f{h)) = {x^yXA\f{g),x^^^XB\f{h)) . 
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Proof. Since E[XA\fig)XB\f{h)] = {xA\f{g),XB\f{h)), we can express Cov j (g), XB\f{h)) as: 

= {xA\fi9),XB\f{h)) - {x^f,XA\fig)){x^f, XB\f{h)) = {xl\fXA\f{g),xlyXB\f{fl)). □ 

The following was first observed in [GSll], and enables controlling probabilistic quantities in 
terms of a geometric quantity, the projection distance to certain subspaces. In particular, it says 
that if we can somehow choose S and /o in such a way that span of '^s\fo is very close to the vectors 



then the variance will be small. 



Lemma 33. Given fo G [k]^° , subsets S,AC [n], and g € [k]^, we have 
Proof. Using Claim 31, we see that: 



E 



f 



^^1/0(5)"^ 



\xA\f0i9) 
\xA\f0i9) 

\xA\fo(.9) 

1^51/0^^1/0(5)11 



X]ll^S|/o(/)f I^AI/, 



fo 



X] II^S|/o (/) f (^0|/,/o ' ^A|/,/o (5))^ 



Y{xs\foif)^XA\fJoi9)f 
f 

3^SuA\foif ° 9)' ^ 



k5|/o(/)ll 



/:xs|^g(/)^0 

/:xs|,„{/)^oll^^l^O^^^II 
5Z^^5|/o(/)> 2:^1/0(5))^ 



'-\\xA\fJo{9)\\') 

(using Claim 28 (c)) 
(using Claim 28 (b)) 
(using Definition 27) 

(using Claim 28 (e)) 
(using Claim 28 (b)) 



(using Notation 29). □ 



B Other Rounding Algorithms 



In this section, we will show how the partial coloring algorithm from [AGll] and 2-CSP algorithm 
from [BRSll] fit into our framework. The main difficulty is that both these algorithms are adaptive. 
In particular, a naive adaptation will have i = i^{r) which is quite undesirable for our faster solver. 
We can easily get around this difficulty by replacing the adaptive seed selection procedure with a 
suitable version of Algorithm 4. 

B.l Partial Coloring of 3-Colorable Graphs 

Theorem 34. Given a 3-colorable d-regular graph G on n nodes, positive real 1 > e > and 
positive integer r, suppose its r*'^ largest eigenvalue of normalized Laplacian matrix, Xn-r, satisfies 



Xr 



4-(5 
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for some positive real S > 0. Then, for the choice of r' = 0{r/5e), Algorithm 5 followed by 
the rounding algorithm as described in [AG 11] will output a partial coloring which colors at least 
(1 — £)i^i^n nodes. Furthermore this algorithm can be implemented in time poly(n)2*-^('''/''^) using 
the faster solver framework. 

The main advantage of our seed selection procedure (which enables the speed-up using our 
faster solver) is that we pick r' nodes all at once, instead of picking them one-by-one in r' steps as 
in [AGll]. We have the following as an immediate corollary of Theorem 34: 

Corollary 35. Given a 3-colorable d-regular graph G, for any positive integer r with \n-r ^ 
^ — 0(1), we can find a partial coloring on j nodes and an independent set of size at least ^ in 
time poly(n)2'-^('') . 

Before we begin the proof of Theorem 34, we will state some simple claims. As the method 
applies for fc-colorable graphs with different parameters, below for clarity we first use k for the 
number of colors, and then later set = 3. 



Claim 36. For any edge (n, v) of G, 



1 2 2 

2 " " k 



In particular, if we use A to denote the normalized adjacency matrix of G, then: 

Tr [X^X{I + ^)] ^ n (^1 - . 



Proof. 

2" " 2 



\ '^(\\xixu{i)\\^ + \\xlx^{i)f + 2{xlxu{i),xlxy{i))^ 



+2{xuii^,x^{i)) - 2{x,j,,Xu{i)){xii,x^{i))\\x,ji\^'^ 

= 1-^ E {\\^u{i)\\^ + \\xv{i)\\^ + 2\\xu{i)f\\x^{i)f) (using (x„(i), x^i)) = 0) 
ie[fc] 

= i-^E(ii^«(^)ii' + ii^''«ii')'- 

ie[fc] 

At this point, observe that X]ie[fc] + ll^f (0 IP)^ is a convex function on ||xu(i)|p and 

||xi,(j)|p's. Since X]. ||xt,(i)|p = J2j Ikf = i* is minimized when ||x„(i)|p = ||x„(j)p = p 
Substituting this into the above expression, we see that: 

1 „ „2 k /2\^ 2 

- x„ + xjr^i-- - =1--. 

2 " " 2\k k 



For the final part, observe that: 



□ 



{u,v}&EiG) 
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Claim 37. Given a graph G and positive integer r, for being the r^^ smallest eigenvalue of 
corresponding normalized graph Laplacian matrix, the following holds: 



Proof. Follows from using the upper bound from Claim 36 on inequality: 

Y,a,{X^X)^^Tr[X^X{I + A)] . 

Claim 38. Assume u is uncolored. Then: 



□ 



Proof. Note that ||2;0jjX„|j(i)|p = — If u is uncolored, then 1 — ^ 

^ for all i E [fc]^, in which case we have: 



E 



l^0|/^"l/{i)l 



□ 



For a subset 5 of vertices of G, we denote by Xg the projection operator onto the orthogonal 
complement of span{X„ \ u £ S}. 

Lemma 39. For coloring f to a subset S sampled with probability \\xs{f)\\'^: 



E 



/ 



El 



Proof. From Lemma 33, we know that E/ ||a;0jyX„ Ifm'l =J:^\\^i^u{i)\\U\\XjXJ\ The 
final inequality follows from the same arguments as in [GSll]. □ 

Proof of Theorem 34- Let 6' = and e' = e6'. By [GS12b], we know that volume sampling of 
r' = 0{r/e') columns yields 



WX^Xuf ^ (1 + e) J] a,{X^X) ^ n(l + e 



1 - 2/k 



2 — At* 



Using Markov inequality, the fraction of uncolored nodes is bounded by: 

A -2/k 2(1 + e') , 
<2„(l + e)^ = J^n (for. = 3). 

For Ar ^ I — this expression becomes {^^§7?^-, which implies 



E [fraction of colored nodes] ^ 1 - = = -e) ^1 + ^/2 



2 + 6 



To prove that the coloring output is legal, notice that for any pair of adjacent nodes {u, v) £ 



E(G), both cannot be larger than 1/2 both at the same time. 



□ 



^This follows from the threshold rounding algorithm used in [AGll] for coloring, which colors u with color i if 
\\x^lf(i)f > 1/2. 



26 



B.2 Approximating 2-CSPs 



Given a 2-CSP problem on variables [n] and labels [k], let G be its constraint graph. For conve- 
nience, we assume G is regular; however all our bounds still hold when G is non-regular. We use 
A to denote G's normalized adjacency matrix and Aj to denote the i^^ smallest eigenvalue of G's 
normalized Laplacian matrix. Finally we will use uf ~ G to denote sampling a constraint with 
probability proportional to the weight of constraint between u and v. 



Embedding. Consider the embedding used in Lemma 5.3 of [BRSll] which is used to con- 
vert k vectors into a single vector. Given a partial assignment / G [k]^ and u G [n] with 
(^M|/(^))ie[/c] KI™-!, we define Xu{f) as the following vector. 

. 1 (4u-.,f(J)r ^^^^ 



Seed Selection and Rounding. We will give only an overview of the seed selection procedure. 
In [BRSll], assuming some lower bound on A^. (in terms of e. A;, where e is the additive approxima- 
tion error), a seed set of r • poly(/c/e) vertices will be picked in as many iterations, one vertex at a 
time. We modify the seed selection to involve fewer adaptive stages, with i = Oik!^ /e^) stages each 
picking 0[r/e) vertices each. Plugging into our general solver then gives a runtime improvement 
as before. 



At i^^ level, we choose a seed set of size 0{r/e), Si, from the matrix X{fi) = [Xu{fi)]u 



en 



where X^s are defined in eq. (15). After choosing seed set Si, we sample an assignment gi G [k]^^ 
(conditioned on fi-i) that satisfies 

where 5/ is defined in eq. (17) and set /j ^ o gi. We repeat the seed selection procedure as 
long as Ef^ > e where is as defined in eq. (16). 

The rounding procedure remains the same — independent labeling for each CSP variable from 
the respective conditional distributions. Formally, for each variable u G [n], we choose a label 
i G [k] with probability (i)|p independently at random. In Theorem 44, we will show that 

1 = ^fr^ ) i-e. seed selection will terminate after choosing at most (. sets. 
Analysis. Let us begin by defining the quantity 

As shown in [BRSll], the above gives an upper bound on the expected extra fraction of unsatisfied 
constraints in the rounded solution compared to the Lasserre SDP optimum (when performing 
rounding after conditioning on assignment /). Therefore, when ej ^ e, we get an additive e-error 
approximation. Our goal is prove (which we will do in Theorem 44) that for i ^ 0{k/e), we must 
have Ef^ ^ e. 
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If we define the quantity 5f measuring the expected total variances of each as 

|2 



5;^E„ J]Var 
ie[fc] 



(17) 



then it is easy to see that £/ ^ k6f by Cauchy-Schwarz. 

We will first relate eq. (16) to the inner products of the embedded vectors Xu{f)- 
Claim 40. E„,^g [(^n(/), ^.(/))] ^ 
Proof. We have 



ij ll^0j/^«|/(*)llll^0j/^f|/(-?')ll Z^ij ll^0j/^M|/(OII ll^0j/^f|/ 

where the second step uses Cauchy Schwarz. Since 

1/2 / \ 1/2 



(18) 



El 



the expected value of the above lower bound (18) for ~ G is at least e'j/k. 
We now upper bound the lengths of the embedded vectors. 

2 

Claim 41. \\Xu{f)f^E^e [k]\\xmfXu\f{'i')\\ ■ In particular, EuWXuiDW^ ^ 6f. 



□ 



Proof. 1||XJ/)||2 =Ei.^^J^^^^4?S^P^^ ^ 



'01/ 



a;«|/(^) 



□ 



Now for fixed / we will upper bound the expected value of 5f^g over g ~ ||a^s|/(5)lP ™ terms of 
the projection distance of the embedded vectors from the subspace spanned by Xy{f) for v £ S. 
(Below, Xs{f)''~ denotes the projection onto the orthogonal complement of span{Xt,(/) | v E S}.) 

Claim 42. Eg^,,, [6f,g] ^ E„^g \\\Xj{f)Xu{f)\\ 



Proof. By Lemma 33, we know that E„ 



. Since X0|j is in the 



span of n^ij by Claim 28(c), n^|^x„|j(i) = n^| y,Xgj^Xu|j(i). Similarly for any v £ S and j G [A;] 



S|/ 01/ "1/^ 



the vector 2;0jja;^|/(j) is in the span of ^s\f- By using the same arguments from [GSll] , namely 

2 2 

the embedding used here preserves linearity, we obtain Yli '^s\f^9\f^ti\fi^) ^ 11^5 (/)"'^«(/)|| • 



Taking expectation over u completes the proof. 



□ 



Using the above, we can prove the main claim about the seed selection procedure, namely that, 
assuming Xr is close enough to 1, the expected variance 5f can be reduced by a geometric factor 
by conditioning on the assignment to a further 0{r/e) nodes. 



= See Claim 24 in the full version of [GSll]. 
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Lemma 43. Given f E [k]^° , positive real e > and positive integer r with A^+i ^ 1 — if 
Ef ^ e then there exists a set of 0{rk'^ / e^)- columns of X{f), S and g € [k]^ such that xg\f{g) ^ 
and: 



Furthermore such S and g can be found in poly{n)k^^^^^^ time using volume sampling [DRIO, 
GS12h] to find S and then enumerating all g's. 

Proof. Let p = e/k, and /i = Eu||Xu|p where for notational convenience we suppress the dependence 
on / and denote X^if) by Xu- Observe that 

^Tr [X'^XA] = Eu.^g{Xu,X,) ^ {e^jkf ^ 
n 

by Claim 40. This impHes ^Tr [X^XL] ^ E„||X„||2 - p'^ = ^- p"^. From Lemma 30 of [GSll], we 
know that volume sampling 0{r/p^) columns from X yields a set S for which: 

, , . , (l/n)Tr [X^XLl , u - p2 

E„||.Y,-A-J^ < (1 + l-!n„(l'-,^^,'o) < (1 + Of"')) frf 

Since p ^ 1, we have (1 - pV2)"^ ^ (l + Ip^): 



^ (1 + 0{p^)) {p - p') (l + Ip'^ ^ (1 + 0{p')) (/. 

^P-n (p2) 

^(5/ - n (by Claim 41) . 

By Claim 42, Eg [dfj ^ E„||X^X„f , which means there exists g for which 6 f^g ^ 5 f — (^j^^ . □ 
We put together everything in the following theorem. 



Theorem 44. For i = O y^jt -seed selection procedure will output a partial assignment fn with 
Proof. Suppose £f. > £ for all i ^ 1. Then by Lemma 43, for each i ^ i: 

which is a contradiction. □ 
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